{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<style>div.container { width: 100% }</style>\n",
    "<img style=\"float:left;  vertical-align:text-bottom;\" height=\"65\" width=\"172\" src=\"../assets/PyViz_logo_wm_line.png\" />\n",
    "<div style=\"float:right; vertical-align:text-bottom;\"><h2>Tutorial 06. Network Graphs</h2></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualizing and working with network graphs is a common problem in many different disciplines. HoloViews provides the ability to represent and visualize graphs very simply and easily with facilities for interactively exploring the nodes and edges of the graph, especially using the Bokeh plotting interface. It can also make use of Datashader for plotting large graphs, and NetworkX for some convenient graph functions:\n",
    "\n",
    "<div style=\"margin: 10px\">\n",
    "<a href=\"http://holoviews.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"../assets/holoviews.png\"/></a>\n",
    "<a href=\"http://bokeh.pydata.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"../assets/bokeh.png\"/></a>\n",
    "<a href=\"http://numpy.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"../assets/numpy.png\"/></a>\n",
    "<a href=\"http://pandas.pydata.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:140px\" src=\"../assets/pandas.png\"/></a>\n",
    "<a href=\"http://networkx.github.io\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:140px\" src=\"../assets/networkx.png\"/></a>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import holoviews as hv\n",
    "import networkx as nx\n",
    "\n",
    "hv.extension('bokeh')\n",
    "\n",
    "%opts Graph [width=400 height=400]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The HoloViews ``Graph`` ``Element`` differs from other elements in HoloViews in that it consists of multiple sub-elements. The ``Graph`` element itself holds the data that indicates whether each node is connected to each other node. By default the element will automatically compute concrete ``x`` and ``y`` positions for the nodes and represent them using a ``Nodes`` element, which is stored on the Graph. The abstract edges and concrete node positions are sufficient to render the ``Graph`` by drawing straight-line edges between the nodes. In order to supply explicit edge paths we can also declare ``EdgePaths``, providing explicit coordinates for each edge to follow.\n",
    "\n",
    "To summarize, a ``Graph`` consists of three different components:\n",
    "\n",
    "* The ``Graph`` itself holds the abstract edges stored as a table of node index pairs.\n",
    "* The ``Nodes`` hold the concrete ``x`` and ``y`` positions of each node along with a node ``index``. The ``Nodes`` may also define any number of value dimensions, which can be revealed when hovering over the nodes or to color the nodes by.\n",
    "* The ``EdgePaths`` can optionally be supplied to declare explicit node paths.\n",
    "\n",
    "#### A simple Graph\n",
    "\n",
    "Let's start by declaring a very simple graph connecting one node to all others. If we simply supply the abstract connectivity of the ``Graph``, it will automatically compute a layout for the nodes using the ``layout_nodes`` operation, which defaults to a circular layout:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Declare abstract edges\n",
    "N = 8\n",
    "node_indices = np.arange(N)\n",
    "source = np.zeros(N)\n",
    "target = node_indices\n",
    "\n",
    "padding = dict(x=(-1.2, 1.2), y=(-1.2, 1.2))\n",
    "\n",
    "simple_graph = hv.Graph(((source, target),)).redim.range(**padding)\n",
    "simple_graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Accessing the nodes and edges\n",
    "\n",
    "We can easily access the ``Nodes`` and ``EdgePaths`` on the ``Graph`` element using the corresponding properties:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "simple_graph.nodes + simple_graph.edgepaths"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Supplying explicit paths\n",
    "\n",
    "Next we will extend this example by supplying explicit edges:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bezier(start, end, control, steps=np.linspace(0, 1, 100)):\n",
    "    return (1-steps)**2*start + 2*(1-steps)*steps*control+steps**2*end\n",
    "\n",
    "x, y = simple_graph.nodes.array([0, 1]).T\n",
    "\n",
    "paths = []\n",
    "for node_index in node_indices:\n",
    "    ex, ey = x[node_index], y[node_index]\n",
    "    paths.append(np.column_stack([bezier(x[0], ex, 0), bezier(y[0], ey, 0)]))\n",
    "    \n",
    "bezier_graph = hv.Graph(((source, target), (x, y, node_indices), paths)).redim.range(**padding)\n",
    "bezier_graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interactive features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Hover and selection policies\n",
    "\n",
    "Thanks to Bokeh we can reveal more about the graph by hovering over the nodes and edges. The ``Graph`` element provides an ``inspection_policy`` and a ``selection_policy``, which define whether hovering and selection highlight edges associated with the selected node or nodes associated with the selected edge. These policies can be toggled by setting the policy to ``'nodes'`` (the default) or ``'edges'``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bezier_graph.options(inspection_policy='edges')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to changing the policy, we can also change the colors used when hovering and selecting nodes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Graph [tools=['hover', 'box_select']] (edge_hover_line_color='green' node_hover_fill_color='red')\n",
    "bezier_graph.options(inspection_policy='nodes')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Additional information\n",
    "\n",
    "We can also associate additional information with the nodes and edges of a graph. By constructing the ``Nodes`` explicitly we can declare additional value dimensions, which are revealed when hovering and/or can be mapped to the color by specifying the ``color_index``. Similarly, we can associate additional information with each *edge* by supplying a value dimension to the ``Graph`` itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Graph [color_index='Type'] (cmap='Set1')\n",
    "node_labels = ['Output']+['Input']*(N-1)\n",
    "edge_labels = list('ABCDEFGH')\n",
    "\n",
    "nodes = hv.Nodes((x, y, node_indices, node_labels), vdims='Type')\n",
    "graph = hv.Graph(((source, target, edge_labels), nodes, paths), vdims='Label').redim.range(**padding)\n",
    "graph + graph.options(inspection_policy='edges')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to supply additional node information without speciying explicit node positions you may pass in a ``Dataset`` object consisting only of various value dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Graph [color_index='Label'] (cmap='Set1')\n",
    "node_info = hv.Dataset(node_labels, vdims='Label')\n",
    "hv.Graph(((source, target), node_info)).redim.range(**padding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Working with NetworkX"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NetworkX is a very useful library when working with network graphs, and the Graph Element provides ways of importing a NetworkX Graph directly. Here we will load the Karate Club graph and use the ``circular_layout`` function provided by NetworkX to lay it out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Graph [tools=['hover'] color_index='club'] (cmap='Set1')\n",
    "G = nx.karate_club_graph()\n",
    "hv.Graph.from_networkx(G, nx.layout.circular_layout).redim.range(**padding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Animating graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like all other elements ``Graph`` can be updated in a ``HoloMap`` or ``DynamicMap``. Here we animate how the Fruchterman-Reingold force-directed algorithm lays out the nodes in real time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Graph [tools=['hover'] color_index='club'] (cmap='Set1')\n",
    "G = nx.karate_club_graph()\n",
    "\n",
    "def get_graph(iteration):\n",
    "    np.random.seed(10)\n",
    "    return hv.Graph.from_networkx(G, nx.spring_layout, iterations=iteration)\n",
    "\n",
    "hv.HoloMap({i: get_graph(i) for i in range(5, 30, 5)},\n",
    "           kdims='Iterations').redim.range(x=(-1.2, 1.2), y=(-1.2, 1.2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Real world graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a final example let's look at a slightly larger graph. We will load a dataset of a Facebook network consisting a number of friendship groups identified by their ``'circle'``. We will load the edge and node data using pandas and then color each node by their friendship group using many of the things we learned above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%opts Nodes Graph [width=800 height=800 xaxis=None yaxis=None]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Graph [color_index='circle']\n",
    "%%opts Graph (node_size=10 edge_line_width=1)\n",
    "colors = ['#000000']+hv.Cycle('Category20').values\n",
    "edges_df = pd.read_csv('../data/fb_edges.csv')\n",
    "fb_nodes = hv.Nodes(pd.read_csv('../data/fb_nodes.csv')).sort()\n",
    "fb_graph = hv.Graph((edges_df, fb_nodes), label='Facebook Circles')\n",
    "fb_graph = fb_graph.redim.range(x=(-0.05, 1.05), y=(-0.05, 1.05)).options(cmap=colors)\n",
    "fb_graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bundling graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Later, in [Working with Large Datasets](10_Working_with_Large_Datasets.ipynb) we will see how the [Datashader](http://datashader.org/) library allows us to render very large datasets efficiently. In this section, we use the algorithms for bundling the edges of large graphs that are available in datashader via HoloViews."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from holoviews.operation.datashader import datashade, bundle_graph\n",
    "bundled = bundle_graph(fb_graph)\n",
    "bundled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datashading graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For graphs with a large number of edges we can datashade the paths and display the nodes separately. This loses some of the interactive features but will let you visualize quite large graphs. If the number of edges is much greater than the number of nodes, using datashader to render the edges still lets you interact with each node for hovering, even though the connections are now drawn as an image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Nodes [color_index='circle'] (size=10 cmap=colors) Overlay [show_legend=False]\n",
    "datashade(bundled, normalization='linear', width=800, height=800) * bundled.nodes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Applying selections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively we can select the nodes and edges by an attribute that resides on either. In this case we will select the nodes and edges for a particular circle and then overlay just the selected part of the graph on the datashaded plot. Note that selections on the ``Graph`` itself will select all nodes that connect to one of the selected nodes. In this way a smaller subgraph can be highlighted and the larger graph can be datashaded to reduce the file size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%opts Graph (node_fill_color='white')\n",
    "datashade(bundle_graph(fb_graph), normalization='linear', width=800, height=800) *\\\n",
    "bundled.select(circle='circle15')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To select just the nodes that are in 'circle15' set the ``selection_mode='nodes'`` overriding the default of 'edges':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bundled.select(circle='circle15', selection_mode='nodes')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Onwards\n",
    "\n",
    "Having seen how to visualize and interactively explore graphical data, we now go on to demonstrate how to visualize and explore a specific domain: [Geographic Data](./07_Geographic_Data.ipynb). While domain specific, geographic data is both very common and typically awkward to handle."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
